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Abstract. Solar storms can have a major impact on the infrastructure 
of the earth. Some of the causing events are observable from ground in the 
Ha spectral line. In this paper we propose a new method for the simulta- 
neous detection of flares and filaments in Ha image sequences. Therefore 

^^ we perform several preprocessing steps to enhance and normalize the im- 

^Sl ages. Based on the intensity values we segment the image by a variational 

< . approach. In a final postprecessing step we derive essential properties to 

^ I classify the events and further demonstrate the performance by comparing 

,^^ our obtained results to the data annotated by an expert. The information 

produced by our method can be used for near real-time alerts and the 

^*D statistical analysis of existing data by solar physicists. 

f"' 1 Introduction 

> 

^^ The activity of the sun and the related space weather have gained wide interest in 

^ the last years. A recently published report by the Royal Academy of Engineering 

O [^ states the drastic impacts a large solar storm can have on the infrastructure 

of the earth and near-earth space. This includes the disturbance of the electrical 

^.H power grid, pipelines and railway networks, satellites and navigation systems. 

^ The cause of such solar storms are mainly solar flares and coronal mass 

CN ejections (CME). Solar flares are an abrupt and enormous release of energy in 

CO a very short time. In a relative small area of the sun a single solar flare can 

j,,^ create an explosion equivalent to several billion hydrogen bombs [5]. CMEs 

on the other hand are correlated with filament eruptions [17 . Both, flares 

J^ and filament eruptions can be observed from ground based observatories in the 

Ha spectral line, which is filtered by a narrow bandpass filter mounted on the 

telescope to create Ha images. 

Flares are characterized in Ha images by a strong brightness increase of 
localized sun areas, reaching the maximum extent and intensity within some 
minutes up to some ten minutes [26 , followed by a gradual decay of intensity. 
^ In figure [l] this event is illustrated. Filaments appear as elongated regions of 

low intensity on the solar disk of Ha images. They have typically about 10% 
of the disk intensity but are still brighter than the sky [32 . Figure [2] shows a 
filament eruption in a Ha image sequence. 

In the Space Situational Awareness^ (SSA) program of the European Space 
Agency {ESA) the University of Graz is implementing a service to provide near 
realtime information about the solar activities. This includes the ground-based 
observation of the sun at the Kanzelhohe Observatory for Solar and Environ- 
mental Research (KSO)^ but also the automatic recognition of solar activities 
in Ha image sequences, which is the aim of this work. The images are taken 
in a period of ten seconds, if the sight conditions are well, have a resolution of 
2048 X 2048 pixels and a radiometric resolution of 12 bits per pixel. 
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^http://www.esa.iiit/0ur_Activities/0perations/Space_Situational_Awareness2 last 
checked on 06.03.2013 
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(a) 08:15:16 



(b) 08:20:09 



(c) 08:23:18 



Figure 1: The cropped image sequence shows an occurring flare. The Ha images 
were taken 27.04.2012 by KSO. 

^^^5 msm'::'^ mm^::< 



(a) 12:50:25 



(b) 13:01:01 



(c) 13:04:06 



Figure 2: The cropped image sequence shows an erupting filament, 
images were taken 09.11.2011 by KSO. 



The Ha 



The remainder of this paper is organized as follows. In section [2] we out- 
line related work dedicated to filament and flare detection in Ha images. Our 
method for the detection and segmentation of these events is presented in sec- 
tion |3] Section |4] demonstrates our experimental results and section [5] concludes 
our work. 



2 Related Work 



The work related to image processing in Ha images for event detection is divided 
in flare detection and filament detection. For the flare detection the earliest 
work was done by [27 . The segmentation is performed by first selecting key 
pixels from the images that have twice the mean intensity and using them as 
an initialization for region growing. This idea was extended by [lOl |23l [22] by 
adding further pre- and postprocessing steps and utilizing machine learning to 
recognize images that show a flare event. 

Filament detection is more difficult and the dedicated work therefore more 
comprehensive. The first published approaches are also based on selecting key 
pixels based on local intensities and region growing [U [15] [131 HH [21] • Another 
common method is based on global and adaptive thresholding [H [231 [25l [28] . 
This methods involve also different preprocessing steps to remove intensity- 
varieties caused by limb darkening and clouds. [28l [3] for example fitted a 
polynomial and subtracted it from the image to obtain a more homogeneous 
solar disk. In [311 [30] an artificial neural network was trained for the binary 
classification, if a pixel belongs to a filament or not, where they used the local 
3x3 neighbourhood as input. 
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3 Method 

The proposed method combines the detection of filaments and flares in one 
framework by segmenting the Ha images. We can divide our method in four 
main steps. In the preprocssing we normalize the images in terms of intensity 
and spatial displacement. We further remove noise and local intensity variations 
with a structural bandpass filter. In the next step we extract and learn features 
to train a classificator that is able to discriminate between filaments, flares 
and background. The result of the classification is further regularized by a 
variational multi-label segmentation approach. In a last postprocessing step 
we assign an ID to each filament and flare, track them through the sequence 
and deduce properties like length, area and intensity distribution from them to 
categorize the events. 

3.1 Preprocessing 

The Ha images taken from ground-based observatories can be disturbed in many 
ways. Variances in the atmosphere have an impact on the exposure time and 
thus the intensity distribution in the images is shifted and stretched. Addition- 
ally the solar disk is not always in the center of the image. Another problem 
arises from clouds that darken partial regions of the image, which has clearly 
an impact on the intensity based detection. 

To address the first mentioned problem we normalize the intensity of the 
input image / to a zero mean and a unit standard deviation as follows 

fn = ^^ (1) 

(J 

where fi denotes to the sample mean and a to the sample standard deviation 
of the image /. 

The solar disk is not always in the center of the image and the position can 
vary in the image sequence. This has influences on the further postprocessing 
and therefore we register every image in the sequence onto the first image. The 
displacement vector u = (1^1,1^2)^ is obtained by minimizing the least squares 
error and linearisation [TSl . On each level L of a Gaussian image pyramid the 
displacement vector is given by 



ul 



E^A^f\l E.,jv/U|v/|, 



■E.,,|v/.|(|v/|-|v^iy 
.E.Jv/,|(|v/|-|v^|). 



(2) 



LE.,,|v/U|v/|, E.,,|v/|^ 

where g and / denote to the normalized reference image and any normalized 
further image in the sequence, respectively and |V/|a: is the partial derivative 
of |V/| in X. We use the gradient magnitude |V/| and |V^| to be more robust 
to image intensity variations. 

As a last step in the preprocessing additive noise and large-scale intensity 
changes, caused by limb darkening and clouds, are removed by applying a struc- 
tural bandpass filter. The TVLi model [2j [3[ [T9[ |20] is a variational denoising 
approach with a total variation ( TV) regularization and a Li data fit term. It 
is able to remove additive noise while preserving sharp discontinuities and is 
further almost contrast invariant. The TVLi model is defined by the following 
optimization problem 

min / d\Vu\ ^X [ \u- f\dx (3) 

^ Jq Jn 
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(a) VI 



(b) V2 



(c) u 



Figure 3: The depicted images show the result of the structural bandpass filter. 
Figure a illustrates the denoised image ^i. The influences of the clouds are 
clearly visible as darker areas on the left and the right of the solar disk. These 
large-scale intensity effects are captured in V2 , but finer structures like filaments 
are not captured by this filter. The result u of the structural bandpass filter is 
shown in figure c. 



where / is the noisy observation, u the solution that minimizes the opti- 
mization problem, and A a free parameter that controls the trade-off between 
the data fidelity (right term) and the total variation regularization (left term). 

In the work of Chan and Esedoglu [8 it is shown that the images obtained 
by changing A generate an interesting scale space. By lowering A objects in 
the image maintain their contrast with respect to their neighbors, while the 
boundaries might gradually smooth out. However, if A reaches a critical value, 
the object suddenly merges with the neighborhood. It is further demonstrated 
in an example that A is related to the radius r of a disk. In the TVLi model 
disks are removed, if the relation < A < ^ is satisfied. 

We utilize this fact in our structural bandpass filter. First, small scaled 
noise is removed from the image by using Ai = 0.9. In a second round we want 
to capture the large scaled intensity variations caused by clouds and the limb 
darkening and use therefore A2 = 0.1. The result of the structural bandpass 
filter is then given if we subtract the second image V2 from the first image vi . 
The algorithm is also shown in Algorithm [T] and the results of the single steps 
are illustrated in figure [3J To minimize the particular TVLi models we use 
the primal-dual algorithm by Chambolle and Pock [7] that can be efficiently 
implemented on the GPU. 

Algorithm 1 Structural Bandpass Filter for an input image / 
Choose Ai > A2 > 

vi = ar; 



■gmin / d|V'i;|+Ai / \v — f\dx 
V Jn Jq 

argmin / d|Vi'| + A2 / \v — f\dx 



V2 

U = Vi — V2 



3.2 Feature Selection and Learning 

On the preprocessed images we have to compute and learn descriptive features. 
An obvious choice is based on the intensity. We created training samples by 
partial annotating 33 from different days of the year 2011 and 2012. From the 
training data, we created histograms for the classes sun spot, filament, flare 
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and background. The estimated distributions have some overlap, especiahy 
between filament and background. To further improve the recognition of fila- 
ments, we employed the fact that filaments are elongated structures and give 
strong response to a Hessian filter [[12]. However, this filter yields also a lot of 
response in the background, near plagues (bright regions) and on fibrils (dark, 
small elongated structures). Therefore this feature added almost no additional 
discriminative information. 

Further, the contrast decreases from the center towards the limb. To incor- 
porate this effect, the distance from the solar disk center to the pixel location 
has proven to be a valuable feature. 

Using the intensity and the distance as features we learned a Gaussian mix- 
ture model of the intensity distribution per class using an expectation maxi- 
mization algorithm J9J. 

3.3 Multi-Label Segmentation 

To enforce a smooth segmentation result we minimize the total length of the 
interface between the classes. This problem is known as the minimal partition 
problem and can formally be written as 

K K 



min - y^Per(l]i) + Ay^ / pi{x)doL 

K 

s.t. vt = yjvti, vti{^ Qj = (Dyi^j 



mm 

- ■ ■ ' (4) 



where Vt denotes the image domain, VeiiVti) the perimiter of the segment i 
andp^ are pointwise weighting functions, in our case the negative log-probabilities 
obtained by the Gaussian mixture models of class i. To further stabilize the seg- 
mentation result, especially for filaments, we compute an exponential weighted 
average of the probabilities over time. 

To compute the solution, we use a convex relaxation of the given problem 
[29] and minimize the scheme by discretization and applying the primal-dual 
algorithm according to [7]. 

3.4 Postprocessing 

The most important task in the postprocessing step is the tracking of the flares 
and filaments in the image sequences. Therefore every connected component in 
the binary segmentation masks is assigned a unique ID. It is common that flares 
occur in two ribbons and that a single filament is seen in the Ha images as many 
interrupted pieces. These parts should be considered as one single component 
and to solve this, we assign the same ID if the distance is less then 25 px for 
filaments and 150 px for flares, respectively. These parameters, as well as the 
others were determined through numerous experiments. 

The IDs for the filaments and flares should also stay the same over time in 
the image sequence. For that tracking issue we propagate the IDs from previous 
images. Each component of the actual image collects ID votes from t previous 
images. The vote count is simply defined by the overlapping pixels for this 
component. Then the filament derives the ID that gained the most votes from 
the previous images. 

For the filaments two problems remain after the segmentation. The first one 
is the border of sun spots. While sun spots have usually a lower intensity than 
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(a) Input image 




(b) Filament detection result 

Figure 4: The right image shows the remaining segmented filaments after the 
postprocessing step, where small components are removed and nearby compo- 
nents are grouped together. 

filaments the border region is exactly captured by the filament class. Therefore 
we remove filament components in a radius of r = 20px around the sun spot 
border. A similar problem arises near and in plagues, the brighter regions in 
the Ha image. These filament detections are more difficult to handle, because 
especially in this region true filaments can exist. We just remove components in 
this active regions that are not elongated by defining a compactness measure- 
ment. The compactness measurement is defined by the area that overlaps with 
the area of a fitted circle. An example result for the filament detection is given 
in figure [4] 

The last step is to derive properties from the components for the categoriza- 
tion of flares and filaments. Flares are classified in Ha images by the area and 
the relative intensity to the solar disk [5]. These values can easily be derived 
from the segmented components. For the filaments the length is an important 
property. To derive the length of the filament components we perform a thinning 
operation [16] . From the remaining pixels of the components we can construct 
a tree and compute the longest path using the Floyd- Warshall algorithm [llj . 



4 Experimental Results 

To evaluate the proposed method we compare the automatically detected events 
to officially registered data by the National Oceanic and Atmospheric Adminis- 
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tration (A/'07474)r] and to the recordings of experts from the KSO, who visuahy 
inspected the Ha image sequences per day. 

For the evaluation of the flare and filament eruption detection, we tested our 
method on the archived images of July 2012. We selected images in a period of 
30 seconds, independent of the image qualities. 

For the flares, we considered all relevant flares of the category 1 and higher. 
This means, the flares exceed a certain area in sun hemispheres [5 . Table fl] 
summarizes the flare detection results in number of true positives, false positives 
and false negatives. We count a flare detection as true positive, if the start and 
end time is in the range of 10 minutes to the official data, the category is correct 
and location differs not more then 10 degrees. The obtained results relate to a 
precision of 1.0, a recall of 0.85714 and a F-score of 0.92307. 



date 


true positives 


false positives 


false negatives 


2012-07-02 


1 








2012-07-03 


3 








2012-07-04 


2 








2012-07-05 


2 








2012-07-06 


3 








2012-07-08 








2 


2012-07-09 


2 








2012-07-10 


2 








2012-07-11 








1 


2012-07-14 


1 








2012-07-29 


1 








2012-07-30 


1 








total 


18 





3 



Table 1: Evaluation of the flare detection of category 1 and above in Ha image 
sequences. The table presents only days, where a flare with importance class 1, 
or higher occurred. 

For the evaluation of the fllament segmentation, we randomly selected one 
image per day of the month July 2012 and an expert annotated the fllaments 
pixelwise. By comparing the created ground truth with the results computed by 
our method we yield an overlap of 0.799825 by measuring the intersection over 
union ratio. For precision and recall we obtain the values 0.92985 and 0.851186, 
respectively and the F-score is equal to 0.888781. It should be noted that the 
fllament segmentation is not unique, due to the lack of unique edges and a low 
contrast to the background. 

The last task is the detection of fllament eruption. Again, we compared our 
data to the registered events of the NAOO and the KSO. We deflned a fllament 
eruption in our method, if we can not track a fllament for 15 minutes in the 
image sequence. Table |2] summarizes the evaluation for the month July 2012. 
Overall, four fllament eruptions occurred in July 2012, whereas the method 
was able to detect all of them correctly. Unfortunately, we detected one false 
positive on 2012-07-03. In this case the method failed, because the fllament 
was not segmented in several images due to clouds. This implies a precision of 
0.80000, a recah of 1.00000 and a F-score of 0.88889. 

The detection of flares and fllament eruptions in the Ha images basically 
share the same problems. The segmentation works flne, until no large clouds 
cover the solar disk, or the events occur not to close to the limb. In those cases 
the segmentation might fail, especially for fllaments. Further, these cases are 
one of the main reasons why a good reconnection of the single fllament parts is 
needed. 



^http : //www . swpc . noaa . gov/f tpmenu/indices/events . html 
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date 


true positives 


false positives 


false negatives 


2012-07-03 





1 





2012-07-08 


1 








2012-07-11 


2 








2012-07-26 


1 








total 


4 


1 






Table 2: Evaluation of the filament eruptions in the Ha image sequences of July 
2013. The table presents only days, where a filament eruption occurred. 

5 Conclusions 

In this paper we presented a new method that combines the detection of fila- 
ments and flares in Ra image sequences. We demonstrated a preprocessing chain 
that performs a normalization, registration, and a structural bandpass filter of 
the image data. From annotated images we learned Gaussian mixture models 
for intensity distributions of the sun spots, filaments, flares and background. 
To regularize the resulting classification, we applied a variational segmentation 
model. After some additional postprocessing steps we can derive properties that 
allow a categorization and further a comparison with official data. The results 
demonstrate that our method is precise in terms of localization and classifica- 
tion. 

Future work will evaluate the method continually on realtime data at the 
KSO. Further, additional research can investigate new features to enhance elon- 
gated structures and enhance the detection results of filaments. Further, the 
grouping of the filament components could be improved, for example by coher- 
ence enhancing diffusion or a strategy based on tensor voting. 
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